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Abstract 

Background:  In  response  to  the  oeeurrenee  of  a  sentinel  event — a  medieal  error 
with  serious  eonsequenees — Eglin  U.S.  Air  Foree  (USAF)  Regional  Hospital 
developed  and  implemented  a  patient  safety  program  called  Medical  Team 
Management  (MTM)  that  was  modeled  on  the  aviation  industry’s  Crew  Resource 
Management  program  and  focused  on  communication,  teamwork,  and  reporting. 
Objective:  To  determine  the  impact  of  a  patient  safety  program  on  patterns  of 
medical  error  reporting.  Methods:  This  study  was  a  retrospective  review  of  1,102 
incident  reports  filed  at  Eglin  USAF  Regional  Hospital  in  Florida  between  1997 
and  2001.  Collected  data  from  the  comparison  periods  (1998  and  2001)  was 
statistically  analyzed  using  the  chi-square  test.  Results:  The  number  of  reports 
submitted  increased  significantly  from  200  for  4,671  hospital  admissions  in  1998 
to  276  for  4,003  admissions  in  2001  (chi-squared  =  28.38,  P  <  0.0001). 

Evaluation  of  incident  severity  showed  172  (86  percent)  near  misses  (no  impact 
on  patient)  in  1998  and  251  (91  percent)  in  2001.  In  1998  there  were  28  (14 
percent)  adverse  events  (patient  minimally  effected)  and  25  (9  percent)  in  2001 
(chi-squared  =  3.302,  P  =  0.069).  Analysis  by  rank  of  person  filing  the  report 
revealed  39  reports  submitted  by  junior  nurses  and  1 1  submitted  by  junior  enlisted 
personnel  in  1998,  while  in  2001  those  numbers  increased  to  75  and  24  reports, 
respectively  (chi-squared  =  6.554,  P  =  0.161).  Conclusion:  This  study  indicates 
that,  since  the  implementation  of  MTM,  there  has  been  a  statistically  significant 
increase  in  the  number  of  reports  filed  at  Eglin  USAF  Regional  Hospital. 
Similarly,  the  severity  of  incidents  shows  an  overall  decline  approaching 
statistical  significance.  Although  there  was  an  increase  in  reporting  from  junior 
team  members,  this  was  not  statistically  significant.  These  findings  suggest  that 
there  have  been  changes  in  the  patterns  of  error  reporting  since  the 
implementation  of  MTM. 


Introduction 

The  Hippocratic  Oath,  a  foundation  of  medical  practice,  urges  practitioners  to 
“first,  do  no  harm.”  However,  the  1999  Institute  of  Medicine  (lOM)  report.  To 
Err  Is  Human:  Building  a  Safer  Health  System^  revealed  that  much  harm  is  being 
done.  This  now  often-cited  report  compiled  statistics  suggesting  that  as  many  as 
98,000  people  may  be  dying  each  year  as  a  result  of  medical  errors. 

Medical  facilities  have  long  had  systems  in  place  to  monitor  errors.  These 
systems,  however,  mostly  consisted  of  the  filing  of  incident  reports  once  an  error 
was  discovered.^’  ^  This  was,  by  design,  a  retrospective  approach  that  often 
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assigned  blame  to  individuals  and  did  little  to  analyze  systems,  identify  trends,  or 
make  recommendations  for  overall  improvements.  With  increasing  media 
attention  and  public  awareness  of  medical  errors,  it  became  clear  that  this  system 
of  monitoring  was  inadequate. 

Patient  safety  has  become  a  central  focus  for  most  medical  institutions  and 
many  new  programs  to  monitor  safety  and  prevent  medical  mistakes  have 
emerged  as  a  result.  The  use  of  new  technologies,  the  employment  of  automated 
systems,  the  introduction  of  system  redundancies,  the  use  of  event  simulation,  and 
the  implementation  of  new  staff  training  are  all  strategies  that  have  been  put  in 
place  in  an  attempt  to  reduce  the  rate  of  errors. 

Like  most  hospitals,  Eglin  U.S.  Air  Force  (USAF)  Regional  Hospital  in 
Florida  had  been  practicing  reactive  error  monitoring.  However,  as  the  result  of  a 
sentinel  event  (a  medical  error  resulting  in  an  unanticipated  death  or  injury)  that 
coincided  with  the  release  of  the  lOM  report,  Eglin  USAF  Regional  Hospital 
decided  to  reconsider  this  system.  A  working  group  was  formed  and  tasked  with 
developing  a  new  strategy  to  identify  and  reduce  medical  errors.  What  resulted  is 
a  patient  safety  program  known  as  Medical  Team  Management  (MTM). 

While  changes  to  staff  training  have  been  a  common  response  to  the  new 
focus  on  patient  safety,*’  ^  MTM  is  unique  in  several  respects.  First,  as  an  Air 
Force  hospital  closely  associated  with  aviation,  this  program  draws  heavily  from 
the  lessons  learned  about  human  error  and  flight  mishaps.''*  The  aviation  industry 
developed  a  safety  program  known  as  Crew  Resource  Management  that  teaches 
fliers  the  principles  of  teamwork,  communication,  stress  management,  and  other 
human  factor  principles  to  prevent  aviation  mishaps.  Crew  Resource  Management 
also  emphasizes  the  need  for  anonymous  reporting  of  near  misses  and  the  removal 
of  blame  as  a  deterrent  to  the  collection  of  accurate  information.*'*’  **  In  addition  to 
incorporating  these  principles,  MTM  is  directed  at  all  members  of  the  medical 
team — physicians,  nurses,  medical  technicians,  and  other  hospital  workers.  As 
miscommunication  has  been  identified  as  the  leading  cause  of  preventable 
medical  errors,*^”*"*  MTM  focuses  primarily  on  facilitating  clearer  communication 
within  and  between  these  groups  to  create  a  safer  patient  care  environment.*^ 

While  course  evaluations  of  MTM  have  been  overwhelmingly  positive,*^  a 
quantitative  look  at  this  training’s  impact  on  communication,  reporting,  or 
severity  of  medical  errors  had  not  been  undertaken.  Because  the  chain  of  events 
that  results  in  a  bad  patient  outcome  is  so  complex,  it  is  difficult  to  analyze 
medical  errors.*’  ^  However,  one  method  of  measuring  the  initial  impact  of  MTM 
is  to  examine  patterns  of  error  reporting.  As  a  routine  component  of  continuous 
quality  improvement  (CQI),  most  medical  facilities  track  errors  and  incidents.  Of 
all  practice  locations,  the  inpatient,  nonintensive  care  unit  setting  is  a  high-volume 
area  with  rapid  turnover  of  patients  and  minimal  staffing  where  the  risk  for 
serious  medical  errors  is  high.*^ 

This  study  hypothesizes  that,  by  focusing  on  improved  communication  and 
the  removal  of  blame  from  reporting  mistakes,  the  initiation  of  MTM  training  will 
result  in  an  increase  in  error  reporting.  As  a  result  of  a  larger  volume  and 
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improved  information  made  available  by  inereased  reporting,  it  is  subsequently 
believed  that  the  severity  of  ineidents  will  deeline  and  that  bad  patient  outeomes 
will  be  averted.  It  is  the  aim  of  this  paper  to  evaluate  the  sueeess  of  MTM  by 
eomparing  rates  of  error  reporting  and  types  of  errors  reported  prior  to  and  sinee 
the  program’s  implementation.  It  will  also  examine  whieh  members  of  the  health 
eare  team  are  filing  reports,  hypothesizing  that  a  eurrieulum  that  emphasizes 
eommunieation  and  empowerment  of  even  the  most  junior  team  members  might 
result  in  an  inerease  in  reporting  from  lower-ranking  nurses  and  medieal 
teehnieians.  Finally,  by  eomparing  the  rates  of  reporting  at  Eglin  USAF  Regional 
Hospital  to  that  of  other  military  medical  facilities  not  exposed  to  MTM  training, 
the  impact  of  this  program  will  be  demonstrated. 

Although  a  direct  cause-and-effect  relationship  may  be  impossible  to 
establish,  finding  significant  differences  in  reporting  rates  will  strongly  suggest 
that  MTM  was  at  least  one  important  factor  in  changing  Eglin  USAF  Regional 
Hospital’s  approach  to  patient  safety. 


Methods 

This  study  is  a  retrospective  review  of  routinely  filed  incident  reports.  As  per 
hospital  operating  instruction,  staff  members  are  required  to  file  an  incident  report 
on  SGQ  Form  1  to  record  any  deviations  from  the  expected  plan  of  care.  At  Eglin 
Regional  USAF  Regional  Hospital,  these  error-tracking  reports  are  processed  by 
the  Quality  Office  (formerly  known  as  the  Risk  Management  Office),  and  each  is 
categorized  as  a  near  miss,  an  adverse  event,  or  a  sentinel  event.  A  near  miss  is  a 
mistake  that  is  caught  and  corrected  before  it  ever  reaches  the  patient.  An  adverse 
event  has  minimal  effect  on  the  involved  patient  and  does  not  result  in  any 
significant  or  permanent  disability.  A  sentinel  event,  however,  is  a  major 
occurrence  resulting  in  an  unexpected  death  or  permanent  disability.  Most 
sentinel  events  are  reportable  and  require  further  investigation  via  a  formal  root 
cause  analysis.  The  total  number  of  reports  received  is  also  used  as  a  surrogate  for 
rate  of  errors  per  hospital  admission.  The  Quality  Office  at  Eglin  USAF  Regional 
Hospital  is  staffed  primarily  by  civilian  contract  employees  and  had  little  turnover 
during  the  period  of  time  covered  by  this  study.  All  incident  reports  included  in 
this  study  were  reviewed  in  an  aggregate  form  with  no  associated  patient 
identifying  information. 

For  this  study,  1,102  incident  reports  filed  by  the  Multi-Service  Inpatient  Unit 
between  1997  and  2001  were  reviewed.  The  Multi-Service  Inpatient  Unit 
provides  care  for  all  medical,  surgical,  and  pediatric  inpatients.  It  is  staffed  by  a 
combination  of  active  duty  Air  Force  and  civilian  contract  nurses  and  medical 
technicians.  It  has  an  average  daily  census  of  26  patients  and  an  average  length  of 
stay  of  48  hours. 

A  historical  review  for  confounding  factors  revealed  that  a  computer  system 
upgrade,  occurring  in  1997,  changed  the  way  report  data  was  processed.  MTM 
training  commenced  in  1999.  Initially,  all  personnel  assigned  to  the  medical  group 
received  the  training  in  scheduled,  large  group  sessions.  Subsequently,  the 
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training  was  directed  at  all  new  hospital  and  clinic  employees  as  an  ongoing  effort 
with  sessions  offered  every  other  month.  By  the  end  of  2001,  over  750  medical 
group  employees,  approximately  91  percent  of  all  assigned  personnel,  had  been 
trained.'^  Given  this,  it  was  determined  to  compare  data  from  incident  reports 
filed  in  1998  to  that  collected  in  2001.  As  a  point  of  comparison,  2001  data  on 
number  and  severity  of  reports  filed  from  all  medical  facilities  in  the  Air  Force 
Materiel  Command  (AFMC)  was  obtained.  These  medical  facilities  are  grouped 
together  solely  by  the  primary  mission  of  the  Air  Force  Base  where  they  are 
located.  Data  from  the  AFMC  was  only  made  available  in  an  aggregate  form 
calculated  per  100  hospital  admissions.  Although  useful  comparison  information, 
its  significance  is  diluted  because  Eglin  USAF  Regional  Hospital  is  a  part  of  the 
Air  Force  Materiel  Command  and  its  statistics  are  also  included  within  this 
dataset. 

Data  pertaining  to  the  total  number  of  reports,  the  assigned  incident  category 
and  severity,  and  the  military  rank/job  title  of  the  person  filing  the  report  was 
entered  for  statistical  analysis.  SPSS  software  (SPSS,  Inc.,  Chicago,  IL)  was  used 
to  analyze  the  data  and  the  Pearson  chi-square  test  was  used  to  compare  different 
datasets.'^ 


Results 

Using  data  collected  from  routinely  filed  incident  reports  from  the  Multi- 
Service  Inpatient  Unit,  the  total  number  of  reports  filed,  the  severity  of  incidents 
reported,  and  the  military  rank/job  title  of  the  person  submitting  the  report  in  1998 
were  compared  to  the  same  parameters  from  2001.  The  total  number  of  reports 
submitted  increased  significantly  from  200  for  4,671  hospital  admissions  in  1998 
to  276  for  4,003  hospital  admissions  in  2001  (P  <  0.0001).  That  is  a  rate  of  6.89 
reports  per  100  hospital  admissions  in  2001  compared  to  4.28  per  100  hospital 
admissions  in  1998  (Figure  1). 

Evaluation  of  the  severity  of  incidents  reported  showed  172  near  misses  (no 
impact  on  patient)  among  200  total  reports  filed  in  1998,  compared  to  251  among 
276  in  2001.  In  1998  there  were  28  adverse  events  (patient  minimally  effected) 
among  200  reports,  and  25  among  276  in  2001  (P  =  0.069;  Eigure  2). 

Standardized  per  100  reports  fded,  that  gives  a  rate  of  90.94  near  misses  and  9.06 
adverse  events  in  2001,  compared  with  a  rate  of  86.00  near  misses  and  14.00 
adverse  events  in  1998  (Table  1). 

Analysis  by  military  rank/job  title  of  the  person  filing  the  incident  report 
revealed  39  of  200  reports  were  filed  by  lieutenants  (junior  nurses)  and  another  1 1 
of  the  200  were  filed  by  junior  enlisted  personnel  (junior  medical  technicians)  in 
1998,  while  in  2001  those  numbers  increased  to  75  of  276  and  24  of  276, 
respectively  (R  =  0.161;  Eigure  3).  Although  absolute  reporting  increased  for  team 
members  of  all  military  ranks/job  titles  from  1998  to  2001,  as  a  percentage  of  all 
reports  filed,  only  lieutenants  (junior  nurses)  and  junior  enlisted  members  (junior 
medical  technicians)  showed  an  increase.  Reports  filed  by  lieutenants  increased 
from  19.5  percent  to  27.2  percent  of  all  reports  filed,  and  from  5.5  percent  to  8.7 
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percent  for  junior  enlisted  members,  although  these  increases  were  not 
statistically  significant. 

Figure  1.  Rate  of  incident  reports  fiied  per  100  hospitai  admissions  in  1998  and  2001 


Figure  2.  Totai  number  of  incident  reports  and  number  categorized  as  “near  miss,” 
“adverse  event,”  or  “sentinei  event”  in  1998  and  2001;  P=  0.069 


Total  Rep  Near  Miss  Adv  Event  Sent  Event 
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Table  1.  Rate  of  near  misses  and  adverse  events  per  100  incident 
reports  filed  in  1998  and  2001;  P=  0.061 


Incident  type 

1998 

2001 

Near  misses 

86.00 

90.94 

Adverse  events 

14.00 

9.06 

Figure  3.  Number  of  personnel  filing  incident  reports  in  1998  and  2001,  sorted  by  military 
rank;  P=  0.161 


□  1998  02001 


Tabie  2.  Comparison  of  reporting  rates  and  incident  severity 
per  100  hospital  admissions  at  Egiin  U.S.  Air  Force  Regional 
Hospital  and  all  medical  facilities  in  AFMC;  P=  0.068 


Egiin 

All  AFMC 

Total  Reports 

6.89 

5.07 

Near  Misses 

6.27 

4.43 

Adverse  Events 

0.62 

0.69 

The  2001  data  on  numbers  and  severity  of  ineident  reports  eolleeted  at  Egiin 
USAF  Regional  Hospital  was  eompared  to  the  same  data  eolleeted  from  all  of  the 
medieal  faeilities  in  the  AFMC.  Per  100  hospital  admissions,  there  were  6.89 
ineident  reports  filed  with  6.27  near  misses  and  0.62  adverse  events  at  Egiin 
USAF  Regional  Hospital,  eompared  to  5.07  reports,  4.43  near  misses,  and  0.65 
adverse  events  for  all  of  AFMC  (P  =  0.067;  Table  2). 


Discussion 

Aviation’s  Crew  Resouree  Management  is  one  of  industry’s  safety  gold 
standards.  Its  foeus  on  attitude  and  leadership,  team  training,  skill  enhaneement 
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with  simulation,  and  reporting  changed  the  culture  of  aviation.^’  Given  that 

aviation  and  health  care  share  many  common  characteristics,  including  high- 
stakes,  complex  environments,  a  team  setting,  and  personnel  with  similar 
personality  traits,  the  concepts  of  Crew  Resource  Management  are  well  suited 
to  application  in  the  health  care  setting.  Medical  Team  Management  has 
incorporated  many  of  the  lessons  learned  from  Crew  Resource  Management.  The 
MTM  curriculum  emphasizes  seven  Critical  Success  Elements: 

•  Daily  Operating  Strategy, 

•  Situational  Awareness, 

•  Workload  Performance, 

•  Available  Resources, 

•  Policies  and  Regulations, 

•  Command  Authority,  and 

•  Medical  Team  Communication. 

These  concepts  are  taught  during  a  4-hour  session  of  lecture,  role-playing, 
interactive  situational  analysis,  and  video  presentations.  In  addition,  MTM  is 
unique  in  its  focus  on  the  entire  medical  team.  Training  has  been  mandated  for  all 
personnel  and  is  included  as  part  of  required  orientation  for  all  new  arrivals  to  the 
medical  group — housekeeper  to  medical  records  administrator,  nurse  manager  to 
senior  physician. 

As  the  breakdown  in  communication  has  been  implicated  as  the  single  leading 
factor  in  medical  mistakes, MTM’s  primary  focus  is  on  improving  team 
interactions  and  clarity  of  communication.  Given  the  multiple  hierarchies  that 
impact  the  medical  team,  particularly  in  the  military  setting,  empowering  team 
members  to  speak  up  and,  sometimes,  question  the  plan  of  care  has  involved  a 
culture  change.  Teamwork  training  has  not,  traditionally,  been  a  part  of  medical 
education,  and  as  a  result,  there  are  many  barriers  to  clear  communication 
between  the  various  contributors  to  the  care  of  a  patient.^*’  MTM  teaches 
communication  by  discussing  verbal  and  nonverbal  cues  and  listening  skills.'^ 
These  concepts  are  implemented  through  several  exercises  and  role-playing 
activities  that  encourage  increased  involvement  from  more  junior  members  of  the 
team. 

In  addition  to  ongoing  training  for  all  medical  group  personnel,  MTM  has 
since  evolved  to  further  facilitate  communication  between  members  of  specific 
care  teams.  This  new  component  of  the  training  has  been  directed  at  the  high-risk 
areas  of  obstetrics,  surgery,  emergency  medicine,  and  the  intensive  care  unit;  it 
involves  eight  small-group  sessions  with  assigned  teams  or  shifts.  The  sessions 
focus  on  identification  of  intra-team  conflicts  and  team-building,  as  well  as  on 
mutually  agreed-upon  problem  areas  and  solution  proposals.  This  phase  of  the 
training  had  not  yet  been  initiated  at  the  time  this  study  was  completed. 
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Like  Crew  Resouree  Management,  MTM  also  emphasizes  the  importance  of 

incident  reporting.  Although  this  is  not  a  new  concept,  its  evolution  into  a 

nonpunitive  tool  is  a  dramatic  change. In  the  past,  acknowledging  mistakes 

has  often  meant  taking  blame.  However,  Crew  Resource  Management,  as  well  as 

recommendations  from  the  lOM,  the  Department  of  Defense,  and  the  Federal 

Government’s  Quality  Interagency  Coordination  Task  Force,  have  all  advocated 

1  20 

the  establishment  of  improved  reporting  systems  as  a  way  to  learn  from  errors.  ’ 

There  are  mandatory  and  voluntary  reporting  systems,  each  of  which  offers 

strengths  at  collecting  certain  types  of  data.^°  Mandatory  reporting  allows  for  the 

collection  of  standardized  data  and  the  identification  of  errors  that  could  and 

should  be  prevented.'’  It  also  facilitates  the  dissemination  of  important 

information  to  the  public  and  holds  health  care  organizations  accountable  for  the 

safety  of  the  care  they  provide.  Voluntary  reporting  programs  are  usually 

anonymous  and,  as  such,  can  often  provide  more  candid  information  that 

ultimately  leads  to  improved  safety  in  processes  and  systems.^'*”^^  A  number  of 

large  medical  and  nonmedical  organizations  have  employed  voluntary  reporting 

systems  to  obtain  improvements  in  their  respective  safety  parameters.^''  Both 

mandatory  and  voluntary  reporting  systems  have  a  role  in  the  overall 

improvement  in  patient  safety.  The  key  to  any  reporting  system  however,  is  in  the 

analysis  and  feedback  of  the  information  gathered  in  the  reports.  The  report 

analysis  often  includes  a  thorough  probe  of  the  root  causes  and  contributing 

factors  associated  with  the  identified  problem,  and  the  feedback  must  present 

leaders  empowered  to  make  change  with  clear  recommendations  for  the 

1  20 

implementation  of  action-oriented  outcomes.  ’ 


Conclusions 

Medical  mistakes  are  a  serious  problem  for  health  care  systems  and  the 
patients  they  treat.'’  '^’  While  medical  facilities  and  health  care  providers  at  all 
levels  strive  to  provide  competent,  compassionate,  and  safe  care,  they  are  not 
currently  achieving  this  goal.  Many  approaches  to  improving  safety  have  been 
initiated  and  many  have  focused  on  improved  training.  MTM  is  a  U.S.  Air  Force 
patient  safety  program  that  is  unique  in  its  origins  from  aviation’s  Crew  Resource 
Management  and  in  its  emphasis  on  communication,  teamwork,  and  reporting. 
The  program  has  been  presented  to  all  employees,  whether  involved  in  direct 
patient  care  or  not,  at  Eglin  USAF  Regional  Hospital,  where  the  program 
originated.  MTM  has  since  been  expanded  to  include  other  Air  Force  and 
Department  of  Defense  medical  facilities  and  has  evolved  into  a  phase  of  team- 
specific  training  in  high-risk  areas.  This  study  suggests  that  changes  have 
occurred  in  the  patterns  of  medical  error  reporting  at  Eglin  USAE  Regional 
Hospital  since  MTM  was  initiated. 

1  10  16  23 

Medicine  is  practiced  in  a  very  complex  environment.  ’  ’  ’  The 
multidimensional  process  by  which  a  medical  mistake  occurs  is  even  more 
complex.  ’  This  complexity  makes  the  analysis  of  errors  and  patterns  of 
mistakes  particularly  difficult  to  analyze.  The  causes  of  mistakes  are  always 
multifactorial  and  therefore,  by  necessity,  so  are  the  remedies.  In  addition  to 
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specific  interventions,  the  media,  general  public  awareness,  and  personal 
experiences  all  impact  on  changes  in  behavior.  Because  of  this,  a  direct  causal 
relationship  between  patient  safety  programs  and  changes  in  specific  safety 
measures  are  difficult  to  establish. 

This  study,  however,  did  demonstrate  a  statistically  significant  increase  in 
medical  error  reporting  after  the  implementation  of  the  MTM  training  program. 

An  increase  in  reporting  is  the  first  step  and  the  key  to  gathering  more  information 
for  analysis,  the  identification  of  trends,  and,  ultimately,  recommendations  for 
changes  in  the  way  health  care  is  practiced.  Similarly,  this  study  also 
demonstrated  a  trend  approaching  statistical  significance  of  an  overall  decline  in 
the  severity  of  the  incidents  reported.  Support  for  the  reality  of  this  trend  and 
changes  in  reporting  that  were  independent  of  external  influences  is  suggested  by 
comparing  the  total  number  of  filed  incident  reports  and  the  severity  of  those 
reports  at  Eglin  USAF  Regional  Hospital  to  other  military  health  care  facilities. 
Finally,  while  the  study  did  show  a  percentage  increase  in  the  number  of  reports 
being  filed  by  the  most  junior  members  of  the  health  care  team,  this  finding  was 
not  statistically  significant. 

Although  these  changes  in  the  patterns  of  medical  error  reporting  at  Eglin 
USAF  Regional  Hospital  are  undoubtedly  the  result  of  many  influences,  the 
implementation  of  the  Medical  Team  Management  training  program  appears  to  be 
one  significant  factor. 
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